Analyzing collaborative processes and learning from hypertext through hierarchical linear modeling

The purpose of this study was to understand how individual and group characteristics interact to produce a rich understanding of domain knowledge. Metanavigation support in the form of prompts was provided to groups of students who collaboratively used a hypertext system called CoMPASS to complete a design challenge. Multilevel analysis techniques were used to understand how the provision of metanavigation support to groups interact with group navigation behavior and learner’s metacognitive awareness of reading strategies to affect individual learning. The findings of this study revealed that providing metanavigation support to the groups contributed positively in enabling students to gain a rich understanding of domain knowledge. Our findings also indicate that there was a significant negative interaction of students’ metacognitive awareness and perceived use of reading strategies and the presence of metanavigation support while interacting with hypertext. Purpose of the study In recent years different methodological approaches have been used to measure and analyze collaborative processes while learning in technology-supported settings. Some of the approaches were: interaction and social network analyses (Jordan & Henderson, 1995; Kreijns, Kirschner & Jochems, 2003; Reffay & Chanier, 2003), various types of discourse analysis (Chinn, O’Donnell & Jinks, 2000), matrix analysis (Wortham, 1999), and content analysis schemes (De Wever, Schellens, Valcke, & Van Keer, 2005; Strijbos, Martens, Prins, Wim, & Jochems, 2005). However, many of these approaches focused on analyzing group discourse. We agree with Naidu and Jarvela (2006) that there is a need to move beyond focusing only on such analyses and direct attention toward understanding how critical attributes of CSCL contexts interact with group collaboration as well as with individual attributes of collaborative learners. Individual, group and context factors affect the types of interactions and the learning outcomes in a collaborative technology-supported setting and need to be taken into account while studying the dynamic process of collaborative learning. Analysis of learning at both the individual and the group unit of analysis is necessary (Stahl, Koschmann & Suthers, 2006) Rummel and Spada (2004) argued that in order to “crack” the complex processes that take place in collaborative contexts we need to work towards developing a “methodological toolbox” which “could support an informed choice of appropriate methods of analysis” (p. 23). Quantitative methods such as multilevel statistical techniques could be useful tools when studying the relationships of variables with different levels and units of analysis. Such methods enable researchers to model the dependencies in the data and obtain more accurate relationships between variables of interest. Recent studies on collaborative learning in technologysupported settings have underlined that there is a “multi-faceted methodological problem” in this area of research (Fischer, Weinberger, & Mandl, 2004) and there is a need for more accurate research methods (in terms of validity and reliability) to assess the impact of learning and working in CSCL settings (Valcke & Martens, 2006). The purpose of this study was to understand how individual and group characteristics interact to produce a rich understanding of domain knowledge. More specifically, we used multilevel analysis techniques to understand how cognitive attributes of collaborative learners might be interacting with group membership to affect learning. We designed and implemented support for navigation (metanavigation support) in the form of prompts to enable groups to think about the processes students use while interacting with online science texts and help them monitor and regulate these processes. Research Context: Integrating CoMPASS in the science classroom This study was a part of an implementation of CoMPASS (Puntambekar, 2006; Puntambekar & Stylianou, 2005; Puntambekar, Stylianou, & Hübscher, 2003; Puntambekar, Stylianou, & Jin, 2001) in sixth grade science classes. During this implementation, students used CoMPASS as a resource to find information and read about the science concepts and principles that were involved in the unit of ‘Simple Machines’. Affordances of CoMPASS CoMPASS is a science hypertext system that has two tightly integrated modes of representation: a textual representation of the content units and a visual representation in a form of concept maps. CoMPASS maps are dynamically constructed and displayed with a fisheye view based on the strength of the relationships among concepts, illustrating graphically the relationships among key ideas in the text (see Figure 1). The maps show the local subnetwork of the domain and where the links lead to, enabling readers to see the relationships among the text units (concepts) and make thoughtful decisions of what paths to follow without getting lost or confused. CoMPASS also supports readers to study a science idea in multiple contexts by changing views (top right of screen in Figure 1). Figure 1. Textual and visual representation of information with ‘work’ as focus In Figure 1 the reader has chosen to read about work in pulley. Work appears as the focal concept in the map and the text related to work appears in the right part of the screen. The concepts that are most closely related to work appear larger and closer to the focus whereas the concepts that are not as closely related to work appear in the periphery. The maps allow for exploration and support students to take multiple investigation paths based on their learning goals at any particular time. Participants The participants in this study were 121 sixth graders in four science classes being taught by two different teachers. The school was located in a university town in Connecticut. The students were from different ethnic backgrounds and academic abilities. Each class was randomly assigned to one of two conditions (metanavigation support, no support). Approximately equal numbers of students were assigned to each condition, with variation being due to uneven class sizes. Students collaborated in groups of three or four while using CoMPASS to solve the “Pulley design challenge”. The groups were formed based on teachers’ perception of students’ academic ability. Teachers decided to form groups of mixed ability levels so that students would benefit from each other during collaboration. The metanavigation support condition included 11 groups of students and the no support condition 15 groups. Procedures The study involved four sessions of 45 minutes that were conducted during the science class period. The first session involved an assessment of students’ metacognitive awareness and perceived use of reading strategies while reading school-related materials through the MARSI (Mokhtari & Reichard, 2002) instrument. This inventory was administered online. The second session started with the presentation of the task. The task was a design challenge that required students to build a pulley device that would lift a bottle of water that weighed 600 grams off a table using the minimum amount of effort. Students were allowed some time to think about the requirements of the task and write down their initial ideas. Then, they were asked to collaborate in groups to plan their quest of finding information to solve the challenge. Groups were asked to read the information that was available for pulleys in the ‘Simple Machines’ unit in CoMPASS. Groups used CoMPASS for approximately 25 minutes. During the third session students were asked to continue their quest of searching information about pulleys in CoMPASS and finalize their pulley system designs. The groups in the metanavigation support condition received metanavigation prompts in a written format to guide their exploration in CoMPASS. Groups were allowed to use CoMPASS for approximately 25 minutes. The fourth session included an assessment of students’ individual science knowledge through a concept map test that was administered in a paper and pencil format. Providing Metanavigation Support Metanavigation support in the form of prompts was provided to the groups in the metanavigation support condition to encourage them to monitor and regulate their navigation strategies in order to gain a rich understanding of science concepts while reading from hypertext. Metanavigation support was based on two indices that were informed by group’s navigation path while interacting with the CoMPASS system. The prompts were contingent upon students’ navigation and were customized for each group. Log file information that captured groups’ navigation path enabled us to assess their navigation behavior and decide what metanavigation prompts would be given to each group. Computer log files recorded information about what science concepts the groups explored while using CoMPASS, how much time they spent on each concept and what navigation tools they used to make their navigation choices. Two main indices from group’s navigation path informed our decision of what type of metanavigation support each group needed: navigation choices and transitions among text units (see Table 1). Specifically, we were interested in whether or not the group members had chosen to read about the science concepts that were relevant to their learning goal and whether the transitions they made among the text units that were available in the hypertext environment would enable them to gain a rich understanding of the domain. For example, did the group make transitions to related concepts while reading about science concepts? Table 1: Group navigation based on log file data Log file information Type Description Concepts visited Non-goal related Goal related Do students visit concepts that are relevant to their learning goal? Transitions No coherence Coherence Do students make transitions to related concepts while reading? Considering the binary state of each of these categories, we could have four different cases, described in the ‘metanavigation support rules’ cells of Table 2, as well as various combinations. Table 2: Conditions for providing metanavigation prompts Metanavigation support rules If choice of non goal-related concepts ⇒ encourage goal-related navigation Navigation choices If goal-related navigation ⇒ encourage integration of science knowledge If transitions are to not related concepts⇒ encourage regulation of navigation behavior to make transitions between text units that are related while reading Transitions If transitions are to related concepts ⇒ encourage integration of science knowledge For example, the log file data of one of the groups indicated that they chose to read about science concepts that were not as relevant for solving the pulley challenge (i.e., ‘kinetic energy’, ‘potential energy’, and ‘power’) and did not read about goal-related science concepts such as ‘mechanical advantage’, ‘distance’, and ‘force’. For example, another group was reading about ‘work’. One possible transition to a related concept would be to read about ‘force’. The metanavigation prompts were aimed at encouraging students to understand the affordances of the navigational aids in CoMPASS and use them to guide their navigation. The prompts encouraged students to (a) think about their goal and (b) to make decisions about which concept to select next. The prompts were designed to help students use the concept maps in CoMPASS to make thoughtful decisions of what paths to follow. As mentioned earlier, the concept maps in CoMPASS showed students the concepts were related to one another and to the topic. Data Sources and Measures Multiple sources of group and individual data were collected over the four sessions. Measures included student’s individual performance in the Metacognitive Awareness of Reading Strategies Inventory (MARSI) and a concept map test. Process measures included log file information that captured group navigation paths during the use of CoMPASS. Pre-Assessment Instruments Students’ metacognitive awareness and perceived use of reading strategies while reading school-related materials was assessed through the Metacognitive Awareness of Reading Strategies Inventory (MARSI) (Mokhtari & Reichard, 2002). MARSI consisted of 30 Likert-type items with a 5-point response format (1=“I never or almost never do this”, 2=“I do this only occasionally”, 3=“I sometimes do this-about 50% of the time”, 4=“I usually do this”, 5=“I always or almost always do this”). An overall total average MARSI score was calculated for each student indicating how often the student uses reading strategies when reading academic materials. Measures during Intervention Computer log files were used to look more deeply into the navigation paths of groups of learners in an attempt to detect differences in approaches to reading and learning from hypertext when providing metanavigation support. Log files recorded information about what science concepts the groups explored while interacting with the CoMPASS system in a chronological order. Two primary dimensions were used for the analysis of group navigation paths. The first dimension was based on whether groups chose to focus on science concepts that were related with their task goal. A goal-relatedness index was calculated by dividing the total number of goal related concepts visited to the total number of concepts visited. The second dimension was based on whether the groups made transitions to related concepts while reading the different text fragments. A transition-relatedness index was calculated by dividing the number of transitions to related concepts to the total number of transitions among concepts. Post-Assessment Instruments A paper and pencil concept map test was used to assess richness of students’ understanding of science concepts. The students were provided with a list of science concepts from which they were asked to create a concept map providing an explanation for each concept, making connections among concepts and stating how they are related. Two aspects of the maps were examined: the explanation provided for the concepts and the explanation provided for the connections among the concepts. Students’ concept maps were analyzed using a rubric that was developed in a study conducted by Puntambekar, Stylianou, and Hübscher (2003). Students’ responses were scored on a scale of 0-3 based on the depth of science understanding that they demonstrated. A score of 0 indicated an incorrect explanation, while a score of 3 indicated a complete and clear explanation for the concept or the connection. A concept ratio was calculated for each student by dividing the score that was given for the explanation of the concepts by the number of concepts included in the concept map. This ratio was a measure of student’s understanding of science concepts. A connection ratio was calculated by dividing the score that was given for the explanation of the connections with the number of connections in the map. This ratio was a measure of the depth of understanding of the relationships among science concepts. Investigations and Data Analyses The main research question that was addressed in this study was: To what extent can concept maps scores (explanations of concepts and explanations of connections) of students be predicted from the presence of metanavigation support while interacting with science texts, their individual metacognitive awareness of reading strategies and the group navigation behavior? In order to analyze the data for this study, multilevel analysis techniques were used (Bryk & Raudenbush, 1992) with the use of the software HLM 6.01 for windows. Multilevel analysis techniques are helpful for taking into account dependencies that occur in datasets that have hierarchical structures. Accounting for such dependencies is especially important in order to reach more accurate estimates of the effectiveness of each independent variable on the outcome variable of interest. For the purpose of the current study, the data were gathered and analyzed on two levels. Level 1 included variables that were gathered on the individual student level; level 2 included variables that were gathered on the group level since the students were nested within groups. Two-level HLM models were tested on two outcome variables. The first outcome variable was the concept ratio (CONCR), a measure of student’s understanding of science concepts. The second outcome variable was the connection ratio (CONNECTR), a measure of the depth of understanding of the relationships among science concepts. For each outcome variable, the HLM analyses were performed in three stages. At the first stage, a null model was tested in which no independent variables were included in the analysis. The results produced by this model were comparable to random effects ANOVA which measured the variance within and between groups. At the second stage, the student-level independent variables were added to the model, while at the third stage the group-level independent variables were added. The independent variables were added to the model based on theory. However, cross-level interactions that were not significant were deleted from the final models. The level 1 data included student level characteristics, which were those of the student’s metacognitive awareness and perceived use of reading strategies while reading school-related materials (MARSI). The level 2 data included group level characteristics which were those of the condition that the students were in (whether they received metacognitive support or not), as well as the two navigation dimensions that were used for the analysis of group navigation paths. The first dimension was the goal-relatedness index (GOALNAV), a measure of whether groups chose to focus on science concepts that were related with their goal. The second dimension was the transition-relatedness index (TRANSNAV), a measure of whether the groups made transitions to related concepts while interacting with CoMPASS. Table 3 includes a more detailed description of the variables used in the analysis. More specifically some descriptive statistics, such as the means, standard deviations as well as the minimum and maximum values of each variable are presented. As shown in Table 3, there was a difference in the averages of the two scores derived from students’ concept maps (concept ratio and connection ratio). The average concept ratio score was higher than the average connection ratio score. It seems that students did not provide many complete and clear explanations for the connections among concepts in their concept map (mean=0.8). The table also shows that the average score of the goal-related navigation index was higher than the average score of the transition-relatedness index. Groups were better in choosing to read about science concepts that were related with their goal than making transitions to related text segments. As far as students’ metacognitive awareness of reading strategies is concerned, it seems that on average students reported that they usually apply reading strategies when reading academic or school related material. Table 3: Description of variables used in the models. Name Description Level Type Minimum Maximum Mean SD CONCR Concept Ratio in Concept Map 1 Outcome 0.00 2.75 1.32 .63 CONNECTR Connection Ratio in Concept Map 1 Outcome 0.00 1.60 0.80 .34 MARSI Metacognitive Awareness of Reading Strategies Score 1 Predictor 1.30 4.70 3.11 .71 CONDITION Indicator of whether the groups received metacognitive support or not 2 Predictor GOALNAV Goal-related Navigation Index 2 Predictor 0.00 1.00 0.66 .31 TRANSNAV Transition-relatedness Index 2 Predictor 0.00 1.00 0.57 .28 Results Predicting Connection Ratio in the Concept Map Test The first analysis that was performed wanted to examine the depth of understanding of the relationship among science concepts. This depth of understanding, also called the connection ratio (CONNECTR) was the first dependent variable that was examined with HLM. Equations 1-3 represent the final model for this sample. Through these models we attempted to explain the differences that students hold in their depth of understanding of relationships More specifically, equation 1 represents the effects of each student’s MARSI score on the CONNECTR variable. This equation examined whether each student’s metacognitive awareness of reading strategies had an effect on their depth of understanding of relationships. Equation 2 represents the group level main effects of CONDITION, TRANSNAV and GOALNAV. This equation examined whether (a) the condition that the students were in (whether they had received support or not); (b) whether each student’s group made transitions to related concepts; and (c) whether each student’s group focused on concepts that were related to their goals, had an effect on their depth of understanding of relationships. Finally, equation 3 represents the interaction between the condition that each group was in with each student’s MARSI score. Level-1 Model (Student level) CONNECTR = β 0+ β1*(MARSI) + R (1) Level-2 Model (Group level) β 0= γ00 + γ01*(CONDITION) + γ02*(TRNSNAV) + γ03*(GOALNAV) + Uo (2) β1= γ10 + γ11*(CONDITION) (3) Table 4. Coefficients of the Connection Ratio Model. Effect Symbol Coefficient Standard error T-ratio Approximate df p-value OVERALL INTERCEPT β0 0.411 0.160 2.560 22 0.018 CONDITION β1 0.692 0.242 2.866 22 0.009 TRNSNAV γ01 0.019 0.136 0.139 22 0.891 GOALNAV γ02 0.324 0.119 2.713 22 0.013 MARSI γ03 0.039 0.041 0.948 81 0.346 CONDITION*MARSI γ11 -0.169 0.068 -2.503 81 0.015 As shown in Table 4, the students who were placed in groups with higher levels of goal navigation, also had higher levels of CONNECTR scores (γ02=0.324, p=0.013). This indicates that the students whose groups chose to focus on concepts that were related to their goals had more depth of understanding of the relationships among the concepts. However, the levels of TRANSNAV that the groups held (whether the groups made transitions to related concepts) did not appear to have any effects on the student’s depth of understanding (γ03=0.019, p=0.891). The results of this analysis have also shown a significant interaction between the condition that the students were in (whether they had received support or not), with the student’s metacognitive awareness (MARSI) (γ11=-0.169, p=0.015). The negative sign of the gamma weight indicates that the students who had received support, but who had lower levels of metacognitive awareness, also had lower levels of depth of understanding. Based on the same relationship, the students who had not received support, but who had high levels of metacognitive awareness also had lower levels of depth of understanding. In order to determine the percentage of variance explained by the models, it was important to estimate the baseline variance that was accounted for in the null model, when no independent variables are added. Based on the unconditional model, the percentage of variance between groups was 11.09%. As a next step, the level 1 predictor (MARSI) was included in the model. Although this variable did not help explain any of the level 1 variance, it was kept in the model in order to test for its interaction with the condition. However, the addition of the MARSI variable did help explain 15.9% of the variance at level 2. Finally, when the final complete model was run, it was able to explain 3.3% of the variance in level 1, and 99.73% of the variance in level 2. Predicting Concept Ratio in the Concept Map Test The procedures that were mentioned above were also performed with the concept ratio (CONCR) as the dependent variable, which measured the student’s understanding of science concepts. As a first step, the same complete model that was used above was tested with CONCR as the outcome variable. Since none of the coefficients were significant however, a stepwise deletion process was preformed. Equations 4-6 describe the final model that was used for this dependent variable. Level-1 Model (Student level) Y = β0 + β1*(MARSI) + R (4) Level-2 Model (Group level) B0 = γ00 + γ01*(CONDITION) + Uo (5) B1 = γ10 (6) Equation 4 represents the level 1 effects of each student’s MARSI score on the CONCR variable. More specifically, this equation examined whether each student’s metacognitive awareness of reading strategies had an effect on their depth of understanding of science concepts. Equation 5 represents the group level main effects of condition, which demonstrated whether the condition that the students were in (whether they had received support or not) had an effect on their understanding of science concepts. Finally, equation 6 demonstrates that the effect of the student’s metacognitive awareness on their understanding of science concepts is fixed, meaning that the relationship between metacognitive awareness and the student’s understanding of science concepts is the same across all groups. Table 5. Coefficients of the Concept Ratio Model. Effect Symbol Coefficient Standard error T-ratio Approximate df p-value OVERALL INTERCEPT β0 0.842 0.226 3.729 24 0.001 CONDITION β1 0.359 0.129 2.784 24 0.011 MARSI γ10 0.100 0.069 1.442 84 0.153 Table 5 describes the effect that each variable had on the dependent variable of interest (CONCR). The independent variable of MARSI was not significant in explaining the student’s CONCR scores (γ10=-0.100, p=0.153). This indicates that the metacognitive awareness of the students did not have anys statistically significant effect on their understanding of science concepts. However, the condition was significant (β1=0.359, p=0.011), indicating that the students whose groups had received support, had higher levels of understanding. In order to determine the percentage of variance explained by this second model, the baseline variance was estimated from the null model, where no independent variables were added. Based on the unconditional model, the percentage of variance between groups was only 7.93%. As a next step, the level 1 predictor (MARSI) was included in the model, which did not help explain any of the variance in any of the two levels. Finally, when the final complete model was run, it was able to explain 0.03% of the variance in level 1, and 96.02% of the variance in level 2. Discussion and Conclusions In this study we used multilevel analysis techniques to understand how critical attributes of a context (provision of metanavigation support to groups while reading from hypertext) interact with group collaboration (group navigation behavior) as well as with individual attributes of collaborating students (metacognitive awareness of reading strategies) to affect individual learning outcomes (understanding of domain knowledge assessed through a concept map test). An overall result that can be concluded from this study is that providing metanavigation support to the groups seems to have contributed positively in enabling students to gain a rich understanding of domain knowledge and have higher scores in the concept map assessment task. The predictive models that were generated using multilevel analysis techniques for both outcome measures in the concept map assessment task, suggest that the variability in concept maps scores (explanations of concepts and explanations of connections) at the group level was accounted for by the presence of metanavigation support. Although the group level variance was very small, for both outcome measures in the concept map test we were able to explain almost all of the group variance. The variability in the scores for the explanations of the connections that each student provided in his/her concept map was accounted by the presence of metanavigation support, the goal related navigation index and by an interaction of his/her MARSI score with the presence of metanavigation support. The presence of metanavigation support and the goal related navigation index had positive significant main effects on the variability of the explanations of connections among concepts in students’ concept maps. Students who collaborated in groups that were given metanavigation support and chose to read about concepts relevant to their learning goal gained a deeper understanding of the relationships among science concepts than students who were not given metanavigation support and did not choose to read about goal-related concepts. Our findings also indicate that there was a significant negative interaction of students’ metacognitive awareness and perceived use of reading strategies while reading from traditional texts and the presence of metanavigation support while interacting with hypertext. If a student had a low MARSI score (reported that he/she is not using frequently reading strategies while reading from traditional texts) the metanavigation support seems not to have helped him/her gain a rich understanding of the domain, as shown in his/her explanations of connections concept map score. Also students who had a high MARSI score but were not provided with metanavigation support did not gain a rich understanding of the domain. Providing metanavigation support to groups whose members reported more frequent use of reading strategies might have stimulated collaborative interactions which led to deeper understanding of the relationships among science concepts. Another finding of the study was that the models that were created using the multilevel analysis techniques were not effective in explaining the variance at the student level. The MARSI score was not a significant predictor of students’ performance in the concept map test (explanations of concepts and explanations of connections). Other variables need to be used to predict the variance at the individual level. Reading comprehension and prior domain knowledge were found to be significant predictors of students’ understanding of domain knowledge when we used regression analyses (Stylianou & Puntambekar, 2004). In this study we chose to add the MARSI variable at the student level because we were more interested in determining how metacognitive awareness of reading strategies interacts with group level characteristics (group navigation behavior and provision of metanavigation support to the groups). Overall, applying Hierarchical Linear Modeling enabled us to model the dependencies in the data (in our case students within groups) and obtain more accurate relationships among the variables of interest. We argue that multilevel analysis techniques can help us unravel some aspects of the complex collaborative processes that take place in a technology-supported setting. For example, the communalities and dependencies that exist in various characteristics of students who are in the same groups violate the assumptions of many parametric test procedures such as Analysis of Variance (ANOVA) and Regression. If we were to use such methods, no inferences of individual behavior would have been made based on the behavior of the group. In order to account for dependencies within the group, proper statistical analyses such as Hierarchical Linear Modeling could be used. It is important, though, to study collaborative processes from multiple perspectives (Hmelo-Silver, 2003; Rummel & Spada, 2004) and apply different methodological approaches (quantitative as well as qualitative methods) to understand the complexity of interactions and learning in such dynamic contexts. Our future research plans are to “crack” the collaborative interactions of groups by examining audiodata of peer interactions during navigation. We plan to focus on groups whose members had high MARSI butnot given support and groups whose members which had high MARSI scores but not given support andinvestigate the negative interaction in the connection ratio predictive model. We will attempt to understand therichness of information contained in a collaborative interaction and identify what aspects characterize goodcollaboration which might lead to in-depth understanding of domain knowledge. Such analyses can contribute toour understanding of the reading comprehension processes employed while interacting with hypertext.Identifying how readers navigate digital texts and what kind of support they need while processing nonlinearinformation will be an important contribution in the hypertext as well as the literacy research fields. ReferencesBryk, A.S., & Raudenbush, S. W. (1992). Hierarchical linear models. Newbury Park: CA: Sage Publications.Chinn, C. A., O’Donnell, A. M., & Jinks, T. S. (2000). The structure of discourse in collaborative learning.Journal of Experimental Education, 69(1), 77 -97. De Wever, B., Schellens, T., Valcke, M., & Van Keer, H. (2005). Content analysis schemes to analyzetranscripts of online asynchronous discussion groups: A review. Computers & Education, 46(1), 6–28. Fischer, F., Weinberger, A., & Mandl, H. (2004). Cracking the nut but which nutcracker to use? Diversity toapproaches to analyzing 23 collaborative processes in technology-supported settings. In Y. B. Kafai, W. A.Sandoval, N. Enyedy, A. S. Nixon, & F. Herrera (Eds.), Proceedings of the Sixth International Conferenceof the Learning Sciences Embracing Diversity in the Learning Sciences (pp. 23-26). Mahwah, NJ:Lawrence Erlbaum Associates. Hmelo-Silver, C. E. (2003). Analyzing collaborative knowledge construction: Multiple methods for integratedunderstanding. Computers and Education, 41, 397-420. Jordan, B., & Henderson, A. (1995). Interaction analysis: Foundations and practice. Journal of the LearningSciences, 4(1), 39-103. Kreijns, K., Kirschner, P., & Jochems, W. (2003). Identifying the pitfalls for social interaction in computer-supported collaborative learning environments: A review of the research. Computers in Human Behavior,19, 335–353. Mokhtari, K., & Reichard, C. A. (2002). Assessing students’ metacognitive awareness of reading strategies.Journal of Educational Psychology, 94(2), 249-259. Naidu, S. & Jarvela, S. (2006). Analyzing CMC content for what? Computers & Education, 46(1), pp. 96 Puntambekar, S. (2006). Learning from Digital Text in Inquiry-Based Science Classes: Lessons Learned in OneProgram. In S. A., Barab, K. E Hay & D. T. Hickey (Eds.) Making a difference: Proceedings of theseventh International Conference of the Learning Sciences (ICLS) (pp. 564-570). Mahwah: NJ. Erlbaum Puntambekar, S. & Stylianou, A. (2005). Designing navigation support in hypertext systems based on navigationpatterns. Journal of Instructional Science, 33, 451-481. Puntambekar, S., Stylianou, A., & Hübscher, R. (2003). Improving navigation and learning in hypertextenvironments with navigable concept maps. Human Computer Interaction, 18(4), 395-426. Puntambekar, S., Stylianou, A., & Jin, Q. (2001). Visualization and external representations in educationalhypertext systems. In J. D. Moore, C. L. Redfield & W. L. Johnson (Eds.) Artificial Intelligence inEducation, AI-ED in the wired and wireless world (pp. 13-22). IOS Press, Netherlands. Reffay, C. & Chanier, T. (2003). How social network analysis cam help to measure cohesion in collaborativedistance-learning. In B. Wasson, S. Ludvigsen & U. Hoppe (Eds.), Designing for change in networkedlearning environments. Proceedings of the International Conference on Computer Support forCollaborative Learning (CSCL) (pp. 343-352). Dordrecht, NL: Kluwer Academic Publishers. Rummel, N. & Spada, H (2004). Cracking the nut – but which nutcracker to use? Diversity in approaches toanalyzing collaborative processes in technology-supported settings. Proceedings of the Sixth InternationalConference of the Learning Sciences, UCLA, Santa Monica (pp. 23-26). Stahl, G., Koschmann, T, & Suthers, D. (2006). CSCL: An historical perspective. In R. K. Sawyer (Ed.)Cambridge Handbook of the Learning Sciences (pp. 409-426.) Cambridge, UK: Cambridge UniversityPress. Strijbos, J. W, Martens, R. L., Prins, F. J., & Jochems, W. M. G. (2005). Content analysis: What are they talkingabout? Computers & Education, 46(1), 29–48. Stylianou, A. & Puntambekar, S. (2004). Understanding the role of metacognition while reading from nonlinearresources. Paper presented at the Sixth International Conference of the Learning Sciences (ICLS), SantaBarbara, CA. Valcke, M & Martens R. (2006). The problem arena of researching computer supported collaborative learning.Computers & Education, 46(1), 1-5. Wortham, D. W. (1999). Nodal and matrix analyses of communication patterns in small groups. In C. Hoadley& J. Roschelle, Proceedings of the International Conference on Computer Support for CollaborativeLearning (CSCL) (pp.681-686). Mahwah, NJ: Erlbaum. AcknowledgmentsThis research was supported by National Science Foundation’s early career grant (CAREER #9985158) to Dr.Sadhana Puntambekar. We thank the students and teachers who participated in the study.